On the consistency of AUC Optimization

AUC (area under ROC curve) is an important evaluation criterion, which has been popularly used in diverse learning tasks such as class-imbalance learning, cost-sensitive learning, learning to rank and information retrieval. Many learning approaches are developed to optimize AUC, whereas owing to its non-convexity and discontinuousness, almost all approaches work with surrogate loss functions. Therefore, the study on AUC consistency is crucial, and the previous study showed that classification calibration is necessary and sufficient for the consistency of AUC. In this paper, we show that, for pairwise surrogate loss of AUC, minimizing the expected risk over the whole distribution is not equivalent to minimizing the conditional risk on each pair of instances. We disclose that classification calibration is necessary yet insufficient for AUC consistency, and provide a new sufficient condition for the asymptotic consistency of learning approaches based on surrogate loss functions. Based on this finding, we prove that exponential loss, logistic loss and distance-weighted loss are consistent with AUC. Then, we derive the q-norm hinge loss and general hinge loss that are consistent with AUC. We also derive the consistent bounds for exponential loss and logistic loss, and obtain the consistent bounds for many surrogate loss functions under the non-noise setting. Furthermore, we disclose an equivalence between the exponential surrogate loss of AUC and exponential surrogate loss of accuracy, and one straightforward consequence of such finding is that AdaBoost and RankBoost are equivalent.